Temporal sentence grounding (TSG) aims to identify the temporal boundary of a specific segment from an untrimmed video by a sentence query. All existing works first utilize a sparse sampling strategy to extract a fixed number of video frames and then conduct multi-modal interactions with query sentence for reasoning. However, we argue that these methods have overlooked two indispensable issues: 1) Boundary-bias: The annotated target segment generally refers to two specific frames as corresponding start and end timestamps. The video downsampling process may lose these two frames and take the adjacent irrelevant frames as new boundaries. 2) Reasoning-bias: Such incorrect new boundary frames also lead to the reasoning bias during frame-query interaction, reducing the generalization ability of model. To alleviate above limitations, in this paper, we propose a novel Siamese Sampling and Reasoning Network (SSRN) for TSG, which introduces a siamese sampling mechanism to generate additional contextual frames to enrich and refine the new boundaries. Specifically, a reasoning strategy is developed to learn the inter-relationship among these frames and generate soft labels on boundaries for more accurate frame-query reasoning. Such mechanism is also able to supplement the absent consecutive visual semantics to the sampled sparse frames for fine-grained activity understanding. Extensive experiments demonstrate the effectiveness of SSRN on three challenging datasets.

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.

The 1$^{\text{st}}$ Workshop on Maritime Computer Vision (MaCVi) 2023 focused on maritime computer vision for Unmanned Aerial Vehicles (UAV) and Unmanned Surface Vehicle (USV), and organized several subchallenges in this domain: (i) UAV-based Maritime Object Detection, (ii) UAV-based Maritime Object Tracking, (iii) USV-based Maritime Obstacle Segmentation and (iv) USV-based Maritime Obstacle Detection. The subchallenges were based on the SeaDronesSee and MODS benchmarks. This report summarizes the main findings of the individual subchallenges and introduces a new benchmark, called SeaDronesSee Object Detection v2, which extends the previous benchmark by including more classes and footage. We provide statistical and qualitative analyses, and assess trends in the best-performing methodologies of over 130 submissions. The methods are summarized in the appendix. The datasets, evaluation code and the leaderboard are publicly available at https://seadronessee.cs.uni-tuebingen.de/macvi.

人群计数是一项回归任务，它估计场景图像中的人数，在一系列安全至关重要的应用程序中起着至关重要的作用，例如视频监视，交通监控和流量控制。在本文中，我们研究了基于深度学习的人群计数模型对后门攻击的脆弱性，这是对深度学习的主要安全威胁。后门攻击者通过数据中毒将后门触发植入目标模型，以控制测试时间的预测。与已经开发和测试的大多数现有后门攻击的图像分类模型不同，人群计数模型是输出多维密度图的回归模型，因此需要不同的技术来操纵。在本文中，我们提出了两次新颖的密度操纵后门攻击（DMBA $^{ - } $和DMBA $^{+} $），以攻击模型以产生任意的大或小密度估计。实验结果证明了我们对五个经典人群计数模型和四种类型数据集的DMBA攻击的有效性。我们还深入分析了后门人群计数模型的独特挑战，并揭示了有效攻击的两个关键要素：1）完整而密集的触发器以及2）操纵地面真相计数或密度图。我们的工作可以帮助评估人群计数模型对潜在后门攻击的脆弱性。

知识图嵌入（KGE）的有效性在很大程度上取决于建模固有关系模式和映射属性的能力。但是，现有方法只能以不足的建模能力捕获其中的一些。在这项工作中，我们提出了一个名为House的更强大的KGE框架，该框架涉及基于两种家庭转换的新型参数化：（1）住户旋转以实现建模关系模式的较高能力；（2）处理复杂关系映射属性的住户预测。从理论上讲，房屋能够同时建模关键的关系模式和映射属性。此外，房屋是对现有基于旋转的模型的概括，同时将旋转扩展到高维空间。从经验上讲，House在五个基准数据集上实现了新的最新性能。我们的代码可在https://github.com/anrep/house上找到。

时间句地接地（TSG）是视频理解的关键和基础。虽然现有方法训练具有大量数据的精心设计的深网络，但我们发现他们可以轻松忘记由于偏移数据分布而在训练阶段的很少出现的情况，这影响了模型概括并导致不希望的表现。为了解决这个问题，我们提出了一个内存增强的网络，称为内存引导的语义学习网络（MGSL-net），它学习并记住在TSG任务中的很少出现的内容。具体而言，MGSL-Net由三个主要部件组成：跨模型互动模块，存储器增强模块和异构注意力模块。我们首先将给定的视频查询对与跨模型图卷积网络对齐，然后利用内存模块在域特定的持久存储器中记录跨模板共享语义功能。在培训期间，内存插槽与常见和罕见的案例动态相关，减轻了遗忘问题。在测试中，可以通过检索存储的存储器来提高罕见的情况，从而产生更好的概括。最后，使用异构注意力模块在视频和查询域中集成增强的多模态特征。三个基准测试的实验结果表明了我们对效率和效率的方法的优势，这在整个数据集上显着提高了准确性，而且在罕见的情况下也是如此。

虽然视觉变形金机在许多视觉任务中实现了骨干模型的优异性能，但大多数都打算捕获图像或窗口中所有令牌的全局关系，这会破坏2D结构中的补丁之间固有的空间和本地相关性。在本文中，我们介绍了一个名为SimVit的简单视觉变压器，将空间结构和本地信息合并到视觉变压器中。具体而言，我们引入多头中央自我关注（MCSA）而不是传统的多头自我关注以捕获高度局部关系。滑动窗口的引入有助于捕获空间结构。同时，SIMVIT从不同层提取多尺度分层特征以进行密集的预测任务。广泛的实验表明，SIMVIT作为各种图像处理任务的通用骨干模型是有效和高效的。特别是，我们的SIMVIT-MICRO只需要3.3M的参数，在Imagenet-1K数据集上达到71.1％的前1个精度，即现在是最小的尺寸视觉变压器模型。我们的代码将在https://github.com/cucasligang/simvit中提供。

深度卷积神经网络（DCNNS）的最新进展显示了热量的性能改进，可见的脸部合成和匹配问题。然而，当前的基于DCNN的合成模型在具有大姿势变化的热面上不太良好。为了处理该问题，需要异构面部额定化方法，其中模型采用热剖面图像并产生正面可见面。这是由于大域的一个极其困难的问题，以及两个模式之间的大姿态差异。尽管其在生物识别和监测中存在应用，但文献中的这种问题相对未探索。我们提出了一种域名不可知论的基于学习的生成对抗网络（DAL-GAN），其可以通过具有姿势变化的热面来合成可见域中的前视图。 Dal-GaN由具有辅助分类器的发电机和两个鉴别器，捕获局部和全局纹理鉴别以获得更好的合成。在双路径训练策略的帮助下，在发电机的潜在空间中强制实施对比度约束，这改善了特征向量辨别。最后，利用多功能损失函数来指导网络合成保存跨域累加的身份。广泛的实验结果表明，与其他基线方法相比，Dal-GaN可以产生更好的质量正面视图。

The Transformer is widely used in natural language processing tasks. To train a Transformer however, one usually needs a carefully designed learning rate warm-up stage, which is shown to be crucial to the final performance but will slow down the optimization and bring more hyperparameter tunings. In this paper, we first study theoretically why the learning rate warm-up stage is essential and show that the location of layer normalization matters. Specifically, we prove with mean field theory that at initialization, for the original-designed Post-LN Transformer, which places the layer normalization between the residual blocks, the expected gradients of the parameters near the output layer are large. Therefore, using a large learning rate on those gradients makes the training unstable. The warm-up stage is practically helpful for avoiding this problem. On the other hand, our theory also shows that if the layer normalization is put inside the residual blocks (recently proposed as Pre-LN Transformer), the gradients are well-behaved at initialization. This motivates us to remove the warm-up stage for the training of Pre-LN Transformers. We show in our experiments that Pre-LN Transformers without the warm-up stage can reach comparable results with baselines while requiring significantly less training time and hyper-parameter tuning on a wide range of applications.

The ability for an agent to continuously learn new skills without catastrophically forgetting existing knowledge is of critical importance for the development of generally intelligent agents. Most methods devised to address this problem depend heavily on well-defined task boundaries, and thus depend on human supervision. Our task-agnostic method, Self-Activating Neural Ensembles (SANE), uses a modular architecture designed to avoid catastrophic forgetting without making any such assumptions. At the beginning of each trajectory, a module in the SANE ensemble is activated to determine the agent's next policy. During training, new modules are created as needed and only activated modules are updated to ensure that unused modules remain unchanged. This system enables our method to retain and leverage old skills, while growing and learning new ones. We demonstrate our approach on visually rich procedurally generated environments.